Top 10 Tips for Peer Reviewers 


) When you receive an invitation to review, the article’s abstract will help you decide 
whether it’s within your area of interest and expertise. Remember to respond 
promptly or else you might delay the process. 


) Keep the contents of any manuscripts you’re reviewing confidential. What’s more, if 
you've submitted similar research of your own, or if you’ve reviewed the article fora 
different journal, let the editor Know there’s a conflict of interest. Agreeing to a review 
for personal gain is not ethical practice. 


) When commenting, make sure your remarks stay within the scope of the paper and 
don't veer off subject. If you’re unclear of the scope, editorial policy, presentation and 
submission requirements, speak to the editor or read the Author Guidelines. 


) Your review should ultimately help the author improve the paper. Be sure to offer 
some constructive feedback, even if your recommendation ends up being to reject. 


) Carefully analyzing and commenting on a manuscript can take a good chunk of time. 
Make sure you have enough time available when taking on a review. 





) Structure your comments by numbering them. It makes the editor’s life a lot easier. 

n You can also divide them into major and minor issues to help authors prioritize 
corrections. Keep comments to authors separate from the confidential ones to 
editors. But make sure your comments to authors correspond to your assessment on 
the confidential review and review checklists/score sheets. 


) If you’re reviewing a paper that’s in English but wasn’t written by a native speaker, 
its good to be tolerant and point out elements that change the meaning, rather than 
commenting on the quality of their English. 














= ) The conclusion will give you a good idea whether the research is an exciting 
Í development within its own field. 


) Editors find it useful if you comment on the number of replicates, controls and 
statistical analyses. Strong statistics are crucial to determining whether the outcome 
is robust. 


i 
) If a paper you’re reviewing is really good and an excellent addition to the existing 
literature, don’t be afraid to say so. 
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How, when and why to say no to a review 


reques t 
Extract from post published on 12" March 2015. 


“The “when” is easy: Do this, as soon as you | 
have established that you cannot perform the 
review on time, or are not scientifically (or 
otherwise) suitable to assess the manuscript: 
editors are exceedingly grateful for quick 
declines. The request to review usually only 
contains the manuscript title and abstract, so 
there is not much reading to be done before 

deciding not to review. And yet, my experience 
in editorial suggests that this is not such an easy decision to get right 
after all. And so to the “why”: 





Andrew Moore 


1. No time to spare 

lf you genuinely don’t have enough time to review, you must decline. 
This innocent question opens out into one of the most contentious 
matters in contemporary peer review and whether/how such a service 
can be formally recognized: peer review is a major burden, and nobody, 
it seems, really has the time. Hence the questions to your self should 
be “Am | prepared to make time to review this piece? And am | 
disciplined enough to stick to the deadline?” If your intentions are noble, 
itis better to bow out early if you really can’t manage it — much better 
than introducing a large delay in the overall peer review process, which 
creates a bad reputation for you; indeed, it can even arouse suspicion 
in the editor and/or author(s) that you actively wish to delay publication 
of the work. 


2. Knowing what you don’t know 
Another unproductive outcome of peer review is a report that damns the 
work in question for unjustifiable reasons. In some cases, this can arise 


Ask about what doesn’t work- a guide for 


peer reviewers 
Extract from post published on 31° March 2015. 


“A negative resultis still a result!” is a common, 
if slightly ironic, utterance that can frequently be 
heard around the ivory towers. | must say that | 
agree with this sentiment, to the extent that | 
think everyone who reviews a scholarly paper 
should be asking about what doesn’t work 
almost as much as what does. It's the things 
that don’t work that set the boundaries for a 
particular method or technique. However, more | 
importantly than that, the unanswered questions as to why something 
doesn't work lay the foundations for future research, and I'd argue that 
this is precisely why peer reviewers should always ask for a few 
“negative results” to be included in every paper they see. 





Richard Threlfall 


Posing the question 

So if you're a referee, how exactly should you ask about what doesn't 
work? As soon as you've read the paper there will no doubt be ideas 
popping into your head, such as “I wonder what would have happened 
if they had tried X or Y?”. The key here is not to be afraid to ask 
obvious questions or to assume that the authors did try X or Y and it 
didn’t work so they simply didn’tinclude it. If the authors have already 
thought about or even tested your suggestion, then it won't be too 
much of a problem to answer your question. If they haven't already 
thought about it, then your question will be helpful in opening up other 
ideas for them. 


a” 


Excha MOES our IDEAS, RESEARCH AND DISCUSSION BLOG 


out of a desire to rubbish fellow scientists — which is, of course, not very 
noble; but more often it comes from being unaware of one’s own lack of 
knowledge in a particular area. Being aware of what we don’t know 
(metacognition) is not so straightforward, and seems to be a significant 
factor in sub-optimal peer review. So, think carefully: do you know what 
you don't know in the field in question? Is that “don’t-know-area” large? 
Then perhaps you shouldn't review this manuscript, even if you think 
you know quite a lot. 


3. Conflict of interest 
Another clear reason for not agreeing to review is if you have a conflict 


of interest- either positive or negative. Perhaps you are not directly 
associated with the publication record of the author(s), but you havea 
close professional connection all the same: or, you have been asked to 
read a pre-submission draft of the manuscript. Most editors check 
potential reviewers against the acknowledgements, but occasional 
oversights are only human, and sometimes a pre-submission reader is 
not acknowledged. 


Finally to the “how”. This also seems very simple: just reply on time 
saying that you can't do the review, and if you know someone who 
would be suitable, scientifically, please let editorial know Should you 
also give an opinion on the paper that you are declining to review? 
Some do. That typically depends on the reason for which you are 
declining to review. If you truly know the field very well, and do not have 
the time to review, you might well be justified in giving an opinion, and it 
might help editorial. However, a thorough, real, peer review Is 
enormously more valuable than a quick comment over email. So, think 
carefully: if you really have something important to say and you think 
that it's worth giving criticism with a constructive aim, you should 
probably agree instead, and try your best to make time for the review” 


Prioritizing the issues 

When asking about the limitations of a study, the skillis to keep your 
requirements as a reviewer in perspective. You may be able to think 
of several things that the authors could try, but on the flip side you 
can't be too demanding or expect authors to stray too far from the 
main scope of their study. Prioritize what you think are the most 
important issue. If you were the editor, could you accept the paper 
without the authors trying X and Y? If not, why not? Make sure to 
clearly state this in your referee report. If you have other ideas but you 
wouldn't necessarily reject the paper if they weren't included, you can 
put these forward as “nice to haves”. These suggestions may be 
informative for the authors in thinking about future work even if they 
aren't studied in depth in the current paper. 


Limitations are very different from inconsistencies 
Where data and conclusions don’t match or one set of results seems 


to contradict another, you have a right as referee to closely examine 
and pose questions until the inconsistency is resolved. In cases where 
inconsistencies are very severe, you are justified in asking for more 
work to be done, but the natural limits of a technique shouldn't detract 
too much from the results and the advances that have been made. 


As an editor, there’s nothing | like to see more in a referee report than 
a reviewer who pushes to know just what the boundaries of a piece of 
work are. Therefore, | encourage everyone who reviews papers to ask 
about limitations and request more data where its necessary, but at 
the same time keep expectations in perspective and remember to 
focus on what has been achieved in a paper, as well as what hasn't. 
Not every method works for every situation, so it's important to know 
where the borders lie. That's why a “negative result” is indeed still 
very much a result. 
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